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ABSTRACT 

Objectives: Pharmacoepidemiological studies are an 
important hypothesis-testing tool in the evaluation of 
postmarketing drug safety. Despite the potential to 
produce robust value-added data, interpretation of 
findings can be hindered due to well-recognised 
methodological limitations of these studies. Therefore, 
assessment of their quality is essential to evaluating their 
credibility. The objective of this review was to evaluate the 
suitability and relevance of available tools for the 
assessment of pharmacoepidemiological safety studies. 
Design: We created an a priori assessment framework 
consisting of reporting elements (REs) and quality 
assessment attributes (QAAs). A comprehensive literature 
search identified distinct assessment tools and the 
prespecified elements and attributes were evaluated. 
Primary and secondary outcome measures: The 
primary outcome measure was the percentage 
representation of each domain, RE and QAA for the 
quality assessment tools. 

Results: A total of 61 tools were reviewed. Most tools 
were not designed to evaluate pharmacoepidemiological 
safety studies. More than 50% of the reviewed tools 
considered REs under the research aims, analytical 
approach, outcome definition and ascertainment, study 
population and exposure definition and ascertainment 
domains. REs under the discussion and interpretation, 
results and study team domains were considered in less 
than 40% of the tools. Except for the data source domain, 
quality attributes were considered in less than 50% of the 
tools. 

Conclusions: Many tools failed to include critical 
assessment elements relevant to observational 
pharmacoepidemiological safety studies and did not 
distinguish between REs and QAAs. Further, there is a 
lack of considerations on the relative weights of different 
domains and elements. The development of a quality 
assessment tool would facilitate consistent, objective and 
evidence-based assessments of pharmacoepidemiological 
safety studies. 



INTRODUCTION 

Several sources of evidence on drug safety 
issues inform Food and Drug Administration 
(FDA) postmarketing safety-related regulatory 



ARTICLE SUMMARY 



Article focus 

■ This article reviews the suitability and relevance of 
available tools for the assessment of the quality of 
pharmacoepidemiological safety studies. 

Key messages 

■ In the context of regulatory safety-related deci 
sion making, quality assessment (ie, assessment 
of the risk of bias), informs the evaluation of 
available evidence and enhances the appropriate 
utilisation of available evidence in assessing the 
balance between benefits and risks of drugs. 

■ The development of a consolidated reporting and 
quality assessment tool would enhance the con 
sistent, transparent and objective evaluation of 
pharmacoepidemiological safety studies. If a tool 
is developed, it is important to determine if there 
is a need for tools tailored for specific study 
designs or if one tool that consolidates these 
considerations might be helpful. 
Key findings from our review of quality assess- 
ment tools include: 

- Many available quality assessment tools do 
not include critical assessment elements that 
are specifically relevant to pharmacoepidemio- 
logical safety studies. 

- Most tools do not distinguish between report- 
ing elements (REs) and quality assessment 
attributes (QAAs). 

- There is a lack of reported considerations on 
the relative weights to assign to different 
domains and elements with respect to asses- 
sing the quality of these studies. 



Strengths and limitations of this study 

■ Strengths 

- A priori creation of a pharmacoepidemiologi 
cal safety study assessment framework. 

- Comprehensive review of the literature. 

- Importance for safety-related regulatory deci 
sion making. 

- Potential to leverage other comparable efforts in 
the comparative effectiveness research arena. 

■ Limitations 

- The purpose and scope of the reviewed tools 
varied greatly. 

- Each tool was reviewed by one reviewer 
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decisions, including spontaneous case reports, registries, 
observational pharmacoepidemiological studies, rando- 
mised controlled trials (RCTs), meta-analyses and other 
sources. Despite the well-known strengths of RCTs in the 
assessment of drug efficacy, specific issues related to the 
design, methodology and transparency of experimental 
studies may limit their ability to fully characterise the 
safety profile of drugs after marketing approval. 1-4 
Pharmacoepidemiological studies, typically observational 
in nature, represent an important hypothesis-testing 
mechanism in the evaluation of drug safety issues sus- 
pected at the time of approval and for new signals emer- 
ging in the postmarket period. In contrast to RCTs, such 
studies, which typically employ broader inclusion criteria 
and leverage claims or electronic medical record data, 
might better reflect the real-life experience of patients. 
Furthermore, pharmacoepidemiological studies afford 
the ability to investigate rare drug-related adverse effects, 
examine risks in patient subpopulations, and assess long- 
term adverse events. Recent health-related legislation will 
increase the availability and adoption of electronic 
healthcare data for such studies. 5 6 

Despite the potential of pharmacoepidemiological 
safety studies to produce robust value-added data, the 
interpretation of findings from such studies can some- 
times be challenging because of their well-recognised 
methodological limitations, including various sources of 
bias and confounding. 7 These limitations also apply to 
the increasing number of comparative effectiveness epi- 
demiological studies. 8-10 The Institute of Medicine's 
recently published report highlights the importance of 
evaluating the quality of evidence and the significant sci- 
entific disagreements that have ensued over the quality 
of studies. 11 Therefore, assessment of the quality of indi- 
vidual studies is essential to evaluating their credibility. 
Transparency in reporting on the design, conduct, ana- 
lysis and results of these studies is a prerequisite for the 
assessment of the quality of the evidence; it is first neces- 
sary to understand the relevant aspects of the study 
design, conduct and analysis, along with the underlying 
assumptions and rationale behind the key scientific 
decisions undertaken by the study team to adequately 
evaluate the credibility of the study. 12 

The FDA recently published a draft guidance on the 
design, conduct and reporting of pharmacoepidemiolo- 
gical safety studies using electronic healthcare data that is 
designed both to enhance the transparency of reporting 
of such studies and to encourage critical scientific think- 
ing regarding their design and conduct. 13 In the future, 
this guidance may improve the credibility of submitted 
pharmacoepidemiological studies by shedding light on 
the pertinent aspects of studies needed to inform the 
evaluation of the internal and external validity of their 
findings. However, even if the bar for transparency and 
reporting of these studies is raised, it will still be necessary 
to evaluate the contribution of these studies to the avail- 
able evidence on an emerging drug safety issue. The 
Grading of Recommendations Assessment, Development 



and Evaluation (GRADE) evidence assessment frame- 
work, used by clinical guideline developers, appropriately 
separates the initial processes of quality assessment and 
the weighing of evidence in the formulation of guideline 
recommendations. 14 15 In the context of regulatory deci- 
sion making concerning safety issues, the use of quality 
assessment tools to assess the risk of bias may add a 
measure of objectivity to the scientific judgment of the 
available evidence and improve the quality of decision 
making. The benefits may not only extend to improving 
decision making by regulators but also by journal editors 
and researchers as well as potentially improving the 
quality of performed studies by stimulating consideration 
of key aspects of these studies during the development of 
the study approach and protocol. 

Although many checklists and scales for the assess- 
ment of epidemiological studies exist, 16-18 most are not 
specifically designed to evaluate pharmacoepidemiologi- 
cal safety studies. Importantly, although the principles of 
epidemiology apply across different fields, there are 
unique challenges in the design, conduct and evaluation 
of epidemiological studies of unintended drug harms 
that warrant consideration of developing a specific vali- 
dated assessment tool (eg, confounding by indication is 
an important challenge that is unique to epidemio- 
logical studies of drug effects). Recent articles have 
suggested the need to develop tools for assessing the 
quality of these studies. 19-22 A recent publication found 
that systematic reviewers and meta-analysts are misusing 
reporting tools like Strengthening the Reporting of 
Observational Studies in Epidemiology (STROBE) due 
to the dearth of validated assessment instruments. 23 

The main objective of this study is to critically evaluate 
the suitability and relevance of available tools for the 
assessment of pharmacoepidemiological safety studies. 
The ultimate goal is to stimulate discussion in the scien- 
tific community about the need for specific tools to 
facilitate the transparent, objective and consistent evalu- 
ation of study quality to inform safety-related regulatory 
decision making. 

METHODS 

For purposes of this paper, quality assessment tools 
are defined as qualitative checklists and/or quantitative 
scales designed to facilitate assessment of the quality of 
epidemiological studies. 

A priori quality assessment framework 

To examine the utility of individual quality assessment 
instruments for the evaluation of pharmacoepidemiolo- 
gical safety studies, we created an a priori assessment 
framework, consisting of domains that include reporting 
elements (REs), and quality assessment attributes 
(QAAs) (table 1). Based on the expert opinion of FDA 
epidemiologists, concepts drawn from the FDA draft 
guidance on such studies, and key findings from seminal 
reviews tools and documents, we established the 
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Table 1 Reporting elements (REs) and quality assessment attributes (QAAs) according to selected domains and percent 
representation among reviewed tools 



Reporting elements (REs) 



Percent 
representation 

(%) 



Quality assessment attributes (QAAs) 



Percent 
representation 

(%) 



A. Research aims 69 

RE 1 : Description of study 69 
objectives, research aims, design, 
study population and data source, 
exposure and outcome 

B. Study population: data sources 84 

RE 1 : Description of participation 77 
rates and discontinuation rates 
RE 2: Description of denominator 1 1 
used for risk assessment 

C. Exposure definition and 61 
ascertainment 

RE 1 : Description of operational 49 
aspects of exposure ascertainment 
and definition 

RE 2: Description of blinding of 21 
outcome status 

RE 3: Selection of exposure risk 5 
window 

RE 4: Description of selected type of 0 

users (incident vs prevalent users) 

RE 5: Description of comparison 10 

group 

D. Safety outcome definition and 69 
ascertainment 

RE 1 : Description of operational 51 
aspects of outcome ascertainment 
and definition 

RE 2: Description of blinding of 25 
exposure status from those 
ascertaining/validating outcomes 
RE 3: Description of follow-up time 16 

RE 4: Description of composite 0 
outcome, if relevant 

E. Analytical approach 85 

RE 1 : Description of analytic 80 

approach, including approaches to 

handle confounding and biases 

RE 2: Description of a priori sample 44 

size/power calculations 

RE 3: Description of data integration 3 

methods, when relevant 

RE 4: Description of measures of 7 

frequency and association 

RE 5: Description of a priori 5 

specifications of subgroup analyses 

F. Results 36 

RE 1 : Description of main results 25 
(unadjusted and adjusted estimates 
and confidence intervals) and 
sensitivity analyses 



QAA 1 : Appropriateness of prespecified 
aims, design, population, exposure and 
outcome to address research aim 



QAA 1 : Extent of participation rates and 
discontinuation rates 
QAA 2: Appropriateness of denominator 
used for risk assessment 



QAA 1: Validity and appropriateness of 
operational aspects used to ascertain 
and define exposure status 



QAA 2: Appropriateness of selected 

exposure risk window 

QAA 3: Appropriateness of selected 

numerator for risk assessment 

QAA 4: Appropriateness of comparison 

group 



QAA 1: Appropriateness/validity of 
outcome ascertainment strategies and 
outcome definition 



QAA 2: Adequacy of follow-up time to 
address research question 
QAA 3: Adequacy of composite safety 
outcome, if relevant 

QAA 1 : Appropriateness of described 
analytic approach 



QAA 2: Appropriateness of approaches 
to handle confounding and biases 
QAA 3: Description of a priori sample 
size/power calculations 
QAA 4: Appropriateness of data 
integration methods, when relevant 



QAA 1 : Consistency of primary, 
secondary and sensitivity analyses and 
consistency of confounding effects with 
known associations 



34 
34 



57 
56 



31 
30 

3 
0 
5 

36 
33 



13 
0 

49 
26 



39 
21 
0 



Continued 



Neyarapally GA, Hammad TA, Pinheiro SP, etal. £M7 Ope/7 201 2;2:e001 362. doi:1 0.1 136/bmjopen-201 2-001 362 



3 



Evaluation of pharmacoepidemiological safety studies 



Table 1 Continued 



Reporting elements (REs) 



Percent Percent 
representation representation 
(%) Quality assessment attributes (QAAs) (%) 



RE 2: Description of patient 15 
disposition 

RE 3: Description of characteristics 18 
of population by comparison group 

G. Discussion and interpretation 36 

RE 1 : Description of findings in 28 
relation to pertinent issues related to 
study design, conduct, limitations 
and power 

RE 2: Description of plausibility of 21 
findings and clinical significance and 
discussion/exploration of alternative 
explanations, comparison with other 
findings 

H. Study team 7 

RE 1 : Description of study team, 7 
conflict of interest, funding sources 



QAA 2: Impact of patient disposition on 
study integrity and generalisability of 
findings 



QAA 1 : Consideration of findings in 
relation to pertinent issues related to 
study design, conduct, limitations and 
power 

QAA 2: Discussion of plausibility of 
findings and clinical significance and 
discussion of alternative explanations, 
comparison with other findings 



QAA 1 : Relevance of study team 
credentials and experience to the 
research area 

QAA 2: Independence of study team and 
funding sources 



20 
18 
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domains pertaining to the design, conduct and analysis 
of pharmacoepidemiological safety studies. Within each 
domain we listed critical elements that need to be con- 
sidered for assessing the validity and interpretation of 
findings from such studies. We made a distinction 
between the REs and QAAs for each domain. This is an 
important distinction as some guidelines are strictly 
developed to discern and evaluate reporting whereas 
other tools are developed to evaluate quality, which 
requires assessment of reporting. The selected elements 
and attributes presented in this table are not intended 
to represent an all-inclusive list of factors, but rather to 
represent critical aspects impacting the internal and 
external validity of pharmacoepidemiological safety 
studies. Of note, although the QAAs necessarily involve 
some subjectivity, their inclusion in an assessment tool 
would facilitate the consistent and objective consider- 
ation and evaluation of key quality attributes across indi- 
vidual studies. As the GRADE framework developers 
have emphasised, although quality assessment is funda- 
mentally subjective, 29 developing a transparent, consist- 
ent approach to assessment of quality is important, 
especially in the regulatory and clinical arena as 
patients, healthcare professionals and sponsors benefit 
from consistent and transparent assessment of available 
evidence for use in decision making. 

Literature search 

A comprehensive literature search of quality assessment 
checklists and scales was performed in MEDLINE, 



EMBASE and Web of Science. Search terms included: 
'assessment', 'tools', 'quality', 'medical research', 'evi- 
dence based research', 'evidence based medicine', 
'meta-analysis', 'randomised controlled trials', 'biological 
product', 'drug', 'pharmaceutical preparation', 'bio- 
logical therapy', 'bias' and 'epidemiology'. A total of 54 
references were retrieved from this search. Two inde- 
pendent reviewers identified potentially relevant abstracts 
(n=26) from the initial literature review (inter-rater reli- 
ability >0.85). Inclusion criteria included quality assess- 
ment tools or relevant reviews developed to evaluate 
RCTs, observational studies or meta-analyses. Exclusion 
criteria consisted of clinical assessment tools, general arti- 
cles or guidance on quality assessment and instruments 
or reviews focused strictly on reporting and not addres- 
sing quality assessment. After reviewing each paper, 
10 relevant tools and 3 seminal reviews were identified; 
13 were excluded based on the exclusion criteria above. 

The most recent, relevant review articles and some 
individual tools for assessing the quality of epidemio- 
logical studies were identified. 25 " 27 The 2007 Sanderson 
review, 25 the most recent, comprehensive review of 
instruments for assessing quality of epidemiological 
studies, served as the starting point. 

We also performed Google Scholar searches to iden- 
tify relevant tools that might not be captured in the 
aforementioned search strategies. Google searches based 
on the first 50 hits included the following terms: 'tool 
quality bias'; 'quality assessment'; 'pharmacoepidemiol- 
ogy'; 'quality assessment epidemiology'; 'tool quality 
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assessment study'; and 'scale quality assessment observa- 
tional studies'. Furthermore, we identified and included 
an European Medicines Agency (EMA) methodological 
checklist 24 because, although it is a reporting checklist, it 
includes domains and considerations designed to inform 
safety evaluations made at a drug regulatory agency. 

The assessment tools identified were reviewed by investi- 
gators based on a priori assessment criteria shown in 
table 1. The percentage of tools assessing the prespecified 
elements and attributes within domains (percent repre- 
sentation) was tabulated. During our review, we documen- 
ted which tools employed some method of validation. 

RESULTS 

Overall, out of 104 tools identified, a total of 96 distinct 
assessment tools, including 82 from the Sanderson review, 
6 from the initial literature review, 7 from the Google 
search and 1 regulatory checklist (ENCEPP checklist) were 
considered for review (figure 1). Out of these, 61 were 
selected for the in-depth review. 14 24 30-87 Tools exclusively 
focused on RCTs, tools focused on clinical assessments, 
and tools that did not include an explicit assessment frame- 
work were excluded (n=35) . 

Representation of a priori assessment domains 
and elements within tools 

The proportion of reviewed tools that included REs and 
QAAs according to each a priori defined domain within 
the framework is shown in figure 1. Table 1 depicts the 
detailed results of our review of the domains, elements 
and attributes. We highlighted the representation of select 
RE and QAA under each domain that may have important 
implications for the assessment of a pharmacoepidemiolo- 
gical safety study. RE and QAA related to research aims 
were addressed in 69% (42/61) and 34% (21/61) of the 
tools, respectively. Regarding the domain assessing study 
population and data sources, 84% (51/61) of the tools 
included RE and 57% (35/61) included QAA (table 1). 



In all, 61% (37/61) of the tools included RE and 31% 
(19/61) included QAA under the exposure definition 
and ascertainment domain (table 1). With respect to 
outcome definition and ascertainment domain, 69% 
(42/61) of the tools included RE and 36% (22/61) 
included QAA (table 1). Out of the 61 reviewed tools, 
85% (52/61) and 49% (30/61) included RE and QAA 
under the analytic approach domain (table 1). Under 
the results domain, only 36% (22/61) and 7% (4/61) of 
tools included RE and QAA, respectively (table 1). 

Of the 61 reviewed tools, 36% (22/61) and 20% (12/61) 
of tools included RE and QAA under the discussion and 
interpretation domain (table 1). Overall, 7% (4/61) of 
the tools addressed the description of the study team (RE) 
and the independence of team and funding sources 
(QAA). 

More than half of the reviewed instruments consid- 
ered REs for domains including research aims, analytical 
approach, outcome definition and ascertainment, study 
population and exposure definition and ascertainment. 
Domains related to the discussion and interpretation, 
results and study team were considered in less than 
40% of the reviewed tools. With the exception of the 
study population/data sources domain, QAA research 
domains were considered in less than half of the assess- 
ment tools, with less than 10% considering results and 
study team domains. Many did not address all pertinent 
domains. 

Most reviewed checklists and scales were not specific- 
ally designed to assess epidemiological studies of 
drug-related harms. Although the EMA framework was 
designed to increase transparency of pharmacoepide- 
miological studies, it focuses on reporting versus quality 
assessment. Our review constitutes the only recent 
comprehensive review of available assessment tools to 
determine if any are appropriate and sufficient for the 
evaluation of pharmacoepidemiological studies. Only a 
small number of the reviewed instruments employed 
some method of validation. 30-36 Most of the tools did 



Figure 1 Percent representation 
by domain of reporting and 
quality aspects considered by the 
assessment tools. 



Study team 
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Results 
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not differentiate between REs and QAAs whereas others 
stratified by these aspects. A small number included 
distinct assessment criteria for different epidemiological 
study designs (eg, case— control, cohort). Tools focused 
more on RE than QAA. Figure 1 displays the percentage 
of checklists and scales that included criteria on the 
assessment domains and elements. 

The proportion of reviewed tools that included REs 
and QAAs according to each a priori defined domain 
within the framework is shown in figure 1 . 

DISCUSSION 

Based on our review, there is no specific tool that is 
adequately designed for the robust evaluation of phar- 
macoepidemiological studies of drug safety. No single 
tool considered all the selected domains and elements 
and most tools failed to address critical evaluation 
elements. Making a distinction between RE and QAA is 
important as even if an element of the study is men- 
tioned in the final report, one must then determine if 
this was appropriate for the specific study in the context 
of the drug safety question. Only a few instruments 
specifically made this distinction as we did in our a 
priori framework. Additionally, important RE and QAA 
were lacking in most of the checklists and scales we 
reviewed which highlights the need for a tool focused 
on the evaluation of epidemiological studies designed to 
evaluate drug-related harms; this need has been previ- 
ously identified by others. 88 

Quality attributes related to exposure definition and 
ascertainment were considered in less than half of the 
assessment tools, with less than 15% including RE and 
QAA pertaining to the comparator group, despite the 
fact that the selection of a comparator is critical for drug 
safety and effectiveness trials and epidemiological 
studies, as the choice of suboptimal comparators can 
provide misleading results. 89 90 Only 30% of the instru- 
ment included quality assessment elements pertaining 
to the validity and appropriateness of the operational 
aspects of exposure ascertainment, and only 36% 
addressed quality attributes of validation of outcome 
ascertainment approaches. These are important facets of 
pharmacoepidemiological safety studies their misclassifi- 
cation may lead to false-negative findings regarding the 
association between a drug and adverse event. 

Only about 40% of the checklists and scales included 
QAA on approaches to handle confounding and biases. 
As observational studies are not randomised, the 
approaches to handle confounding and bias are of para- 
mount importance. 7 91 92 This is an important limitation 
of most tools because there are often uncertainties 
regarding results from pharmacoepidemiological studies 
due to the limitations of electronic healthcare data and 
complex nature of the practice of medicine. 92 Only a 
small percentage of tools (28% RE; 18% QAA) included 
elements on the consideration of study findings in the 
context of the design, conduct, limitations and statistical 



power despite the fact that these elements are essential 
in assessing implications of study findings. 

Some of the tools we reviewed were designed as 'all- 
purpose' assessment instruments for evaluation of clinical 
trials and observational studies, while others focused on a 
particular study design (eg, case-control, cohort) . It may 
be useful to create one consolidated, validated tool for 
evaluating observational pharmacoepidemiological safety 
studies focused on general reporting and quality attri- 
butes; tools for the specific study designs, that is, case 
—control and cohort studies, may also be useful due to 
some of the unique aspects of these designs. By creating 
such a tool, regulatory agencies, clinical guideline develo- 
pers, and clinicians could consistently evaluate studies and 
for decision making. The creation of this instrument 
could be led by an independent group of expert metho- 
dologists, perhaps with input from multiple stakeholders, 
including regulators and professional organisations. 

Although we did not address weighing of importance 
of different domains and elements based on their relative 
impact on study contribution to the available streams of 
evidence, this may be an important consideration in the 
formulation of an assessment tool. Also, it is not clear if 
numerical scores are helpful in assessing the quality of 
epidemiological studies, as when numerical scores were 
used to evaluate systematic reviews or meta-analyses of 
such studies, they did not produce valid results. 93 The 
appropriate tradeoff between the utility of a checklist or 
scale for review and the comprehensiveness of the evalu- 
ation elements has yet to be determined. This is compli- 
cated by the lack of validation of most of the available 
tools. Before these issues can be addressed, it is first 
necessary to engage in a broader discussion of the utility 
of such assessment tools in the evaluation of pharmacoe- 
pidemiological safety studies. It is worth noting that crit- 
ical assessment elements of pharmacoepidemiological 
studies focused on effectiveness may be different than 
those focused on safety; however, pharmacoepidemiologi- 
cal comparative effectiveness studies focus on both com- 
parative safety and benefits associated with drugs and 
thus such elements are not mutually exclusive. 94 Thus, it 
is important to consider the potential to leverage current 
efforts to create a validated assessment tool (GRACE 
checklist 95 ) for observational comparative effectiveness 
pharmacoepidemiological studies. 

Our review has some limitations. The purpose and scope 
of the checklists and scales we reviewed, varied greatly. 
Although we conducted a comprehensive review, there 
may be tools that we were unable to access or that were 
published after our search. If a reporting or quality assess- 
ment element contained some aspects of the element, we 
counted this as full representation, even if not all the 
important sub-elements were included. Each checklist or 
scale was reviewed by one study team member; repeating 
the evaluations via a second reviewer was deemed unneces- 
sary at this stage as the primary goal of the review was to 
obtain a broad understanding of the utility of available 
assessment tools in evaluating pharmacoepidemiological 
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safety studies based on a preliminary assessment frame- 
work. Some factors that may be increasingly relevant in 
future studies, such as electronic health records with lin- 
kages to other data sources like outpatient claims, health 
information exchanges or personal health records, were 
not included in our framework but may be included in a 
future validated instrument. Guidelines and checklists pub- 
lished after the time period of our review have included 
some elements that may be important for future linked 
studies which may leverage the increasing availability of 
these data sources. 96 

In the evaluation of many emerging safety issues, phar- 
macoepidemiological safety studies are discussed and may 
influence safety-related decision making. However, often 
the quality-driven contribution of such studies is not dis- 
cussed in a consistent way. The development of an assess- 
ment tool based on expert input may facilitate consistent, 
evidence-based quality assessment of such studies and the 
subsequent determination of their value based on evaluat- 
ing the impact of bias on the robustness of a study results, 
and the interpretation of its findings, within the context of 
the specific drug safety issue. The framework we devel- 
oped may serve as a foundation for future development of 
such an instrument. Efforts to improve the evaluation of 
the contribution of pharmacoepidemiological safety 
studies would be consistent with the FDA's focus on 
strengthening regulatory safety science. 97 If after further 
consideration and discussions with stakeholders develop- 
ment of a tool to evaluate epidemiological data for drug 
safety is pursued, it would be necessary to first determine 
the scope of the assessment tool as well as steps for its com- 
prehensive validation. Further, relevant aspects of the 
design and analysis of pharmacoepidemiology studies 
should be considered (we refer the reader to some helpful 
references). 98-100 Importantly, such a tool would be 
intended to complement, and not replace, expert clinical, 
methodological and statistical expertise necessary to com- 
plete a robust evaluation and determination of the contri- 
bution of a specific pharmacoepidemiological safety study 
to the available evidence for regulatory decision making. 
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